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Abstract. Each of the 16 federal states of Germany has its own school system and also its own pol¬ 
icy to integrate informatics, computer science or ICT into this system. Till present there aren't any 
tests of students’ knowledge on a nation-wide level. Therefore nation-wide or international con¬ 
tests currently offer the only opportunities to compare the knowledge of the participating students 
on a large scale level. By evaluating the overall performance of the students of different federal 
states, we were able to compare the effectiveness of different approaches of teaching Informatics 
at secondary schools. 
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1. Introduction 

In Germany the school systems, the subjects and the curricula are very different from 
state to state. At the early 70ies of the last century, nearly all federal states of Western 
Germany (BRD) started to integrate informatics lessons into the curricula of their sec¬ 
ondary schools, especially at the most demanding type of these, called Gymnasium. Most 
of them restricted informatics to eligible courses in the highest grades 11-13, e.g., the 
state with the largest population. North Rhine-Westphalia (NW). After the German Re¬ 
unification in 1990, most of the eastern German states simply copied an education system 
of one of the western states, like Brandenburg (Br) did. 

Consequently, at the beginning of the new millennium, there was no German state 
(except Saxonia) that had installed a regular subject of informatics before grade 11. At 
this time, the state of Bavaria (Ba), the second largest state regarding population, de¬ 
cided to introduce a regular, compulsory subject of informatics into the time table of its 
405 Gymnasiums, attended by about 370.000 students currently (Hubwieser and Muller, 
2000). The new subject should start with grade 6 in autumn 2004. 

The new subject of Informatics in Bavaria has three stages: in grade 6 and 7, all 
students of the Gymnasium, currently about 45.000 in each grade, have to attend one 
compulsory lesson per week; in grade 9 and 10, there are 2 lessons per week, compulsory 
for all students that have chosen natural science/technology as their direction of study 
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(currently about 17.000 per grade); in grade 11 and 12 the students can choose an eligible 
Informatics course of 3 lessons per week. 

The decision of the government was the consequence of the persistent lobbying of the 
Faculty of Informatics of the TU Munich, which had made its first proposals concerning 
the conception of such a subject already in 1996 (Hubwieser et al., 1996). The content 
and the structure of the curriculum follow more or less the proposals we have made (see 
Breier and Hubwieser, 2002; Hubwieser, 2003, 2004, 2006, 2007). Additionally we have 
developed a series of textbooks for the subject that has been introduced by about 50% of 
the schools (Hubwieser, 2008). 

Meanwhile (in summer 2010), the new subject has reached grade 11, thus it seems 
to be high time to evaluate how it is working. In order to evaluate the notion and opin¬ 
ion of the teachers towards the subject, we have conducted a large scale teacher survey 
in December 2009 (Miihling et al., 2010). Concerning the performance of the students, 
there are no nation-wide performance tests in informatics until now. Thus nation-wide 
competitions like the Bundeswettbewerb Informatik or the Informatik-Biber (Bundeswet- 
tbewerb Informatik, 2010) represent the only opportunities to compare the knowledge of 
the students of different states. 

In this paper we present the results of an investigation on the data of the Informatik- 
Biber contest of autumn 2007, which will show that the Bavarian students are performing 
better than those of comparable other states, as far as they have attended the new subject. 


2. The Bebras Contest 

The annual Informatik-Biber competition is the German issue of the international Bebras 
Contest (see Dagiene, 2008, 2010; Dagiene and Futschek, 2008, 2009). It is performed in 
all German states and in all types of secondary schools in three different age levels: level 
1 for grade 5-7 (in Germany the students of this level are 10 to 13 years old generally), 
level 2 for grade 8-10 (13 to 15 years) and grade 3 for grade 11-13 (15 to 19 years). 

Officially the students don’t need to have any obligatory prerequisite knowledge for 
the competition. Every participant had to answer 15 multiple choice questions in three 
degrees of difficulty: Five easy, five average and five difficult. In total the test is containing 
30 different questions for each of the 3 age groups listed above. Every age group has to 
answer a different set of questions. Some items are the same for different age groups, but 
with different degrees of difficulty. 

We examined the 30 questions of the Informatik-Biber 2007 concerning the overlap¬ 
ping of its topics with the Bavarian curriculum. It turned out that 9 out of the 30 test items 
were covered surely in the preceding lectures, 10 eventually and 11 hardly. Nevertheless 
contests like the Informatik-Biber allow students to use their knowledge gained in lessons 
in informatics (Dagiene, 2010). 

The participants start with 45 points. Any correct answer adds 6 points, when it is one 
of the easy questions, 9 points for average questions, 12 points for difficult questions, any 
incorrect answer subtracts 2 resp. 3 resp. 4 points. Not answering at all is rated with 0 
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points. By this way the participants may score from a maximum of 180 points down to a 
minimum of 0 points. 

The competition is open to all types of secondary schools. Nevertheless, we restricted 
this evaluation to the Gymnasium, because all other types either don’t offer any system¬ 
atic courses in informatics or are not comparable over different states regarding their 
structure. 


3. The Situation in Autumn 2007 

Because of the huge number of participants and the variety of subjects and curricula, we 
decided to restrict the evaluation to three states. We chose Bavaria (Ba) and NW, firstly 
because these are the two largest states of Germany and had the most participants in the 
contest, secondly because of their very different approaches concerning the integration 
of informatics in secondary schools. Additionally we chose Brandenburg (Br), because it 
has many participants on the one hand and as its school system is very similar to NW on 
the other hand. 

As explained above, the Bavarian Gymnasium has established a compulsory subject 
of informatics from grade 6 to 10 and eligible courses in grade 11 and 12. At the time 
of the evaluated Informatik-Biber 2007 the subject had just started in grade 9. Thus the 
students of the age levels 5-7 and 8-10 had partly attended a regular subject of informat¬ 
ics (see ’’Schulordnung fur die Gymnasien in Bayern (Gymnasialschulordnung - GSO),” 
2010b; ’’Schulordnung fur die Gymnasien in Bayern (Gymnasialschulordnung - GSO),” 
2010a). We roughly estimate that about 66% of the age group 5-7 and 33% of the age 
group 8-10 did so, while the participants of the 3rd age level (grade 11-13) did not have 
any systematic lessons in informatics. 

In North Rhine-Westphalia informatics is not a regular subject, but in the grades 7 to 9 
(with focus on grade 8) informatics issues are integrated into the curricula of mathematics 
and other subjects in the extent of one hour per week. In grade 9 to 10 there are optional 
courses in informatics, whereas in 11 to 13 eligible advanced courses can be chosen with 
two or even more hours per week (Weeger, 2007). 

In Brandenburg the situation is quite similar to North Rhine-Westphalia. Thus we can 
estimate that in North Rhine-Westphalia and Brandenburg, the participants of the first two 
age levels (5-7 and 8-10) did have education in informatics integrated in other subjects 
like mathematics, while a (unknown) part of the 3rd age group had attended advanced 
courses in informatics (Weeger, 2007). 


4. Evaluating the Scores 

The results of the international Bebras contest were available in a database table. We 
exported the test scores from the German section by using SQL statements, and analyzed 
them using a spreadsheet. 
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Table 1 

Numbers of participants 


Column # 

1 

2 

3 

4 

5 

6 

7 

Federal State 

Participants 

Participants 

Gymna¬ 

sium 

Participants 
grade 5-7 
Gymnasium 

Participants 
grade 8-10 
Gymnasium 

Participants 
grade 11-13 
Gymnasium 

Proportion 

Participants 

Gymnasium 

Proportion 
of students 
of the state 
to all 

Germany* 

Baden- 

566 

486 

55 

268 

163 

3.0% 

14.1% 

Wurttemberg 

Bavaria 

5601 

4543 

3028 

1338 

177 

28.4% 

15.8% 

Berlin 

510 

53 

12 

9 

32 

0.3% 

3.6% 

Brandenburg 

2174 

1887 

467 

532 

888 

11.8% 

2.4% 

Bremen 

0 

0 

0 

0 

0 

0.0% 

0.8% 

Hamburg 

150 

150 

56 

66 

28 

0.9% 

2.0% 

Hessen 

843 

187 

11 

65 

111 

1.2% 

7.4% 

Mecklenburg- 

2193 

1394 

384 

576 

434 

8.7% 

1.5% 

Vorpommern 

Niedersachsen 

1133 

790 

396 

248 

146 

4.9% 

10.6% 

North Rhine- 

4737 

3205 

835 

1356 

1014 

20.0% 

24.5% 

Westphalia 

Rheinland-Pfalz 

1187 

890 

208 

244 

438 

5.6% 

5.2% 

Saarland 

29 

29 

2 

18 

9 

0.2% 

1.2% 

Saxony 

1598 

1292 

457 

573 

262 

8.1% 

3.4% 

Saxony-Anhalt 

674 

529 

249 

193 

87 

3.3% 

2.0% 

Schleswig- 

37 

36 

10 

7 

19 

0.2% 

3.7% 

Holstein 

Thuringia 

531 

537 

224 

158 

155 

3.4% 

1.9% 

Total 

21963 

16008 

6394 

5651 

3963 

100.0% 

100.0% 


Unfortunately the database table did not contain any information about the type of 
school the participants attended. In order to identify the participants as students of a 
Gymnasium, the name of the school was analyzed. The results are shown in Table 1. 
The number of participants from the Gymnasiums shown there should be the lower limit 
of the real number, because not every school name of a Gymnasium contains this term 
explicitly. 

From the extracted test scores we calculated the sample mean and the 5% confidence 
interval, as plotted in Figs. 1-4. 

4.1. Numbers of Participants 

The numbers of participants were varying strongly over the states. For some states the 
number was even too low to compare them validly with the other states (Table 1). 

Column 6 in Table 1 shows the proportion of participants from Gymnasiums of a cer¬ 
tain state compared to the number of participants from Gymnasiums over all states, while 
column 7 shows the proportion of the students (over all school types) of a state compared 
to the students of all Germany. By comparing these two proportions we can estimate, how 
well a state is represented in the contest. This shows that, e.g., Bavaria is strongly over¬ 
represented, while, e.g., Baden-Wurttemberg is strongly underrepresented. Of course this 
is only a first, rough approximation, because we supposed, that the proportion of students 
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Fig. 1. Sample mean of all grades with 5% confidence interval. 
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Fig. 2. Sample mean of grades 5-7. 


that attend a Gymnasium is nearly the same over the states (Bildungsberichterstattung 
Autorengruppe, 2010). 


4.2. Comparison of Test Scores 


As shown in Table 2 and Fig. 1 the sample mean over all three age groups for all partic¬ 
ipants is 95.9 +/- 0.5, for participants from Bavaria 98.1 +/- 0.8, for participants from 
North Rhine-Westphalia at 99.6 +/- 1.1 and for participants from Brandenburg at 89.7 
+/- 1.4. Both the participants from North Rhine-Westphalia and Bavaria are significantly 
better than the average of all high school students and significantly better than the average 
of the students from Brandenburg. 
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Fig. 3. Sample mean of grades 8-10. 
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Fig. 4. Sample mean of grades 11-13. 

Table 2 
Overview 



Overall 

Bavaria (Ba) 

Nordrhein 

Westfalen (NW) 

Branden-burg (Br) 

All Grades 

95.9 

98.1 

99.6 

89.7 


+/-0.5 

+-0.8 

+-1.1 

+-1.4 

5-7 

92.7 

95.4 

91.6 

89.1 


+-0.7 

+-1 

+-1.9 

+-2.8 

8-10 

97.3 

104.3 

101.6 

88.2 


+-0.8 

+-1.7 

+-1.6 

+-2.8 

11-13 

99.1 

98.7 

103.4 

90.9 


+-1.0 

+-4.5 

+-2.0 

+-2.1 




Comparing the Efficiency of Different Approaches to Teach Informatics 


245 


The difference between the Bavarian and North Rhine-Westphalia’s high school stu¬ 
dents is not significant. 

The picture is different, if we consider the three age groups separately. In grade 5- 
7 the means of the results from North Rhine-Westphalia and Brandenburg don’t differ 
significantly from the overall mean, while the Bavarian results are significantly better 
(Fig. 2). 

In grade 8-10 the sample mean of Bavaria and North Rhine-Westphalia is significantly 
higher than the overall result, with the Bavarian participants (although not significantly) 
better than the North Rhine-Westphalia’s. Remarkable is the poor performance of the 
Brandenburg students (Fig. 3). 

In grade 11-13 the situation is different again. The sample mean of Bavaria doesn’t 
differ significantly from the overall result, while the sample mean of North Rhine- 
Westphalia is significantly better. The participants from Brandenburg scored significantly 
worse. Particular attention hast to be directed to the fact that from Bavaria only 177 stu¬ 
dents participated in this age group, while from North Rhine-Westphalia there were 1014, 
from Brandenburg 888 and in total 3963 (Fig. 4). 

While in North Rhine-Westphalia the age groups 8-10 and 11-13 are causing the 
above-average performance, the Bavarian results arise from the grades 5-7 and 8-10. 
Brandenburg is only within 5-7 on the same level with the overall results, in 8-10 and 
11-13 the sample means were significantly worse. 


5. Interpretation 

The strong performance of Bavarian students in the age groups 5-7 and 8-10 might be 
interpreted as an effect of the introduction of the compulsory subject informatics. Regu¬ 
lar and autonomous lessons thus seem to result in better contest scores than integrating 
informatics issues into other curricula. 

The strong performance of North Rhine-Westphalia in 8-10 and 11-13 are presum¬ 
ably a result of an effective education in informatics by elective, basic and advanced 
courses in informatics. 

It is difficulty to argue with the differences of the curricula themselves because other 
influences might be found in external conditions for the implementation of the Informatik- 
Biber in schools such as improved IT facilities or more or less assistance in preparing the 
test, which were not gathered by the contest. Informatics as an autonomous subject has 
surly more weight and possibilities to initiate supporting settings for learning and doing 
informatics in schools. 


6. Conclusion 


In order to investigate the reasons for the differences in performance further research is 
required. Nevertheless it seems plausible, that students in those groups perform signif¬ 
icantly better, where the newly introduced compulsory and regular subject informatics 
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took place. This is a strong argument supporting regular lessons in computer science as a 
subject of its own instead of integrating informatics topics into other subjects. 

Additionally it will be thrilling to evaluate the results of Informatik-Biber 2008 and 
2009, as in Bavaria, the compulsory education in computer science reached grade 10 
respectively grade 11 at this time. We have just acquired the data of these tests in order 
to compare the results. 

We have to thank Wolfgang Pohl from the Bundeswettbewerb Informatik, Kirsten 
Schltiter from the University of Erlangen-Nuremberg and the team of the Informatik- 
Biber in Germany for disclosing of test results. 
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Skirtingp metodii veiksmingumo palyginimas mokant informaciniu 
technologiju vidurineje mokykloje 

Christoph STEER, Peter HUBWIESER 

Kiekviena is 16 Vokietijos federaliniij zemiij turi autonomines mokyklu sistemas, taip pat in¬ 
formaciniu technologiju, IKT integravimo \ mokymu strategijas. Iki siol nera atlikta jokiu val- 
stybinio masto mokiniu ziniu tyrimu. Todel tik visos salies ar tarptautiniai tyrimai suteikia gali- 
mybiu didelio masto lygmeniu palyginti mokiniu zinias. Siame straipsnyje analizuojamas infor- 
matikos technologiju mokymo vidurinese mokyklose skirtingais metodais veiksmingumo palygini¬ 
mas, jvertinant bendru skirtingu federaliniu zemiu mokiniu mokymusi. 



